Research Statement Reinforcement Learning

Despite decades of research into artificial intelligence, today’s computer systems still require extensive manual effort to program and to customize to our needs. These current limitations motivate my research, which aims to create autonomous agents that can adapt to the complexities and uncertainties of the real world. I hope to develop what I believe is the missing ingredient: knowledge representations and algorithms that are simultaneously grounded in experience yet abstract enough to permit effective reasoning. Much of my work builds upon the foundation of reinforcement learning, a computational framework for learning behaviors from experience. Its emphasis on interaction data over prior knowledge leads to algorithms that are designed to handle arbitrary environments but that learn too inefficiently for many actual applications. My doctoral thesis grants learning agents inductive biases that fit general forms of real-world structure. Hierarchy plays a particularly important role in how I allow agents to generalize more effectively from finite data to infinite environments. Apart from my contributions to fundamental learning algorithms, I also have hands-on experience with promising application domains. I worked on a practical hybrid algorithm for combining reinforcement learning with human expertise in an autonomic computing setting, and I designed a hierarchical behavior-execution framework for a team of soccer-playing AIBO robots. My experience with these projects has convinced me that even these complex environments are now within the reach of learning algorithms that map effectively between low-level data and high-level concepts. This theme of abstracting deep structure from surface experience pervades my research.